Computer-based analysis of seller performance

ABSTRACT

A performance analysis system analyzes the performance of affiliate sites that provide links to specific items in an electronic catalog, and identifies catalog items that can be listed by such affiliate sites to improve performance. An association mining component analyzes transaction data attributable to specific categories of affiliate sites to identify items that are frequently purchased in combination by users of such sites. The detected item associations are used to evaluate, for a given affiliate site, whether significant disparities exist between the expected and actual sales quantities of specific items. The results of the analysis are incorporated into affiliate-specific performance reports, which may include specific recommendations for improving performance. The disclosed methods may also be used to analyze the performance of, and provide recommendations to, online sellers within an marketplace or auction system.

PRIORITY CLAIM

This application is a continuation of U.S. application Ser. No.12/853,207, filed Aug. 9, 2010, which is a division of U.S. applicationSer. No. 10/987,309, filed Nov. 12, 2004 (now U.S. Pat. No. 7,797,197),the disclosure of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to computer-implemented methods foranalyzing the performance of an affiliate site, an online seller, oranother entity, and for generating recommendations for improvingperformance.

2. Description of the Related Art

Online merchants commonly set up affiliate programs through whichindividuals and other business entities (“affiliates”) can refercustomers in exchange for some form of compensation. Typically, eachaffiliate operates a web site that includes one or more links to a website of the online merchant. These links are commonly tagged with anidentifier of the affiliate, so that transactions originating from suchtagged links can be credited to the appropriate affiliate. Thus, forexample, if a user follows a link from an affiliate web site and thenmakes a purchase from the merchant's web site, the merchant may pay theaffiliate a referral fee, such as commission.

Some affiliate programs support the ability for affiliates to list, andto provide links to the merchant's catalog pages for, specific items(products, services, etc.) that are available for purchase from themerchant. For instance, an affiliate web site associated with aparticular subject area, such as cooking, may provide tagged links tothe item detail pages of selected products associated with that subjectarea. Each such link may be displayed in conjunction with a description,which may or may not include an explicit recommendation, of thecorresponding item. Thus, each affiliate may effectively operate avirtual store that lists specific items for sale, and which relies on anonline merchant to process and fulfill orders for items listed in thevirtual store. One example of how such a system can be implemented isdescribed in U.S. Pat. No. 6,029,141, the disclosure of which is herebyincorporated by reference.

With affiliate programs that support links to specific items, thesuccess of a given affiliate may depend heavily on the affiliate'sability to select items that are of interest to users of its web site,and to adequately promote these items on the affiliate site. In manycases, however, affiliates do not have the resources or tools needed toascertain the item preferences of their respective users, or tootherwise evaluate the effectiveness of their web sites. As a result, anaffiliate may, for example, fail to list items that are of particularinterest to its users, or may fail to display such items in asufficiently prominent location on the affiliate web site.

SUMMARY OF THE DISCLOSURE

A system and methods for are disclosed for analyzing the performance ofaffiliate sites, and for identifying items that can be listed on suchaffiliate sites to improve performance. The system is particularlyapplicable to affiliate programs in which affiliates are permitted, orrequired, to select specific items to list on their sites. The systemmay also be used to analyze the performance of, and providerecommendations to, online sellers within an online mall, an onlinemarketplace, or an online auction system.

In some embodiments, an association mining component programmaticallyanalyzes transaction data associated with specific categories, types orgroups of affiliate sites or sellers to identify items that arefrequently purchased in combination. The association mining componentmay, for example generate association rules of the form X1→X2 (P), whereX1 and X2 are mutually exclusive sets of items, and P represents theconditional probability that a purchaser of X1 will also purchase X2. Aseparate set of association rules may be generated for each category,type, or group of affiliate web sites.

The item associations detected by the association mining component areused to evaluate the performance of specific affiliate sites or sellers,and to identify items that are likely to be of interest to their usersor customers. In one embodiment, performance is analyzed by applying theapplicable set of association rules to transaction data associated withthe affiliate site or seller to calculate expected sales quantities ofspecific items. These expected item quantities are compared to thecorresponding actual item quantities to evaluate whether significantdisparities exist.

The results of the performance analysis may be incorporated into areport that is conveyed by email, personalized web page, or othercommunications method to the corresponding affiliate or seller. Thisreport may include specific recommendations of items to be listed ordisplayed more prominently by the affiliate or seller. For instance, ifa disparity is detected in which the affiliate's expected quantity for aparticular item is significantly greater than the actual quantity, thereport may suggest adding this item to the affiliate's web site.

In embodiments for analyzing the performance of online sellers, aseparate set of association rules may be generated for each of aplurality of categories of online sellers, based on the sales data ofthe online sellers in the respective category. The performance of eachparticular online seller may then be evaluated by comparing the actualsales data of the particular online seller to the set of associationrules for the corresponding seller category.

Neither this summary nor the following detailed description purports todefine the invention. The invention is defined by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the components of a system for analyzing affiliateweb site performance, and for providing associated item recommendationsto affiliates, in accordance with one embodiment of the invention.

FIG. 2 illustrates a sequence of steps that may be performed by theassociation mining engine of FIG. 1 to generation association rules.

FIG. 3 illustrates a sequence of steps that may be performed by thedisparity detection module of FIG. 1 to analyze the performance of aspecific affiliate site.

FIG. 4 illustrates one example of a type of performance report that maybe generated by the report generation module of FIG. 1.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS I. Overview (FIG. 1)

FIG. 1 illustrates the components of a system for analyzing affiliateweb site performance, and for generating associated recommendations, inaccordance with one embodiment of the invention. The system includes anonline sales web site 30 that is part of a commerce system 32. Theonline sales site 30 provides functionality for users to browse and makepurchases from an electronic catalog of items 34.

The items represented in the electronic catalog 34 may, for example,include new and/or used physical products that are shipped to customers,digital products that are transferred electronically to customers,subscriptions, tickets for travel and entertainment events, services,and/or other types of items that can be purchased online. These itemsmay be offered for sale on the online sales site 30 by a single businessentity (e.g., a retail merchant) or a collection of business entities.(As used in this detailed description and the claims, the terms“purchase” and “sell,” and their derivatives, are not limited totransactions that involve a transfer of ownership of the item beingpurchased or sold, but rather also encompasses rentals, licenses andleases of items.) Many thousands or millions of different items may berepresented in the electronic catalog 34. Each such item may bedescribed in the electronic catalog by a corresponding item detail pagethat provides functionality for ordering the item.

In the particular embodiment shown in FIG. 1, the commerce system 32includes components (discussed below) for implementing an affiliateprogram, and for making “personalized” (affiliate-specific)recommendations regarding catalog items to list on affiliate web sites.As discussed below under the headings EXTERNALLY MANAGED AFFILIATEPROGRAMS and WEB SERVICE IMPLEMENTATIONS, some or all of thesecomponents may alternatively be part of a separate site or system, whichmay be operated by a separate business entity. This may be the casewhere, for example, the operator of the online sales site 30 partiallyor wholly out-sources the function of managing the affiliate program toan affiliate program service provider.

In the embodiment shown in FIG. 1, the online sales web site 30 includesan affiliate registration module 36, which may be implemented as acollection of web pages and associated program code. Using this module36, a web site operator (or an entity planning to set up a web site) canregister online to operate as an affiliate of the online sales site 30.As part of the registration process, the affiliate registration module36 typically assigns a unique affiliate ID to the registrant, andsupplies the registrant with instructions for incorporating this ID intolinks 38 to pages of the online sales site 30. These links 38 arereferred to herein as “affiliate links.” The online sales web site 30may also provide a link generation tool that generates HTML, or othercoding, for embedding affiliate links 38 within web pages of affiliatesites 40.

During or following the online registration process, each registrant maybe prompted to specify one or more categories, or other attributes, thatcharacterize the registrant's existing or proposed web site 40 orbusiness. This may be accomplished by prompting the registrant to selectfrom a list, browse tree, or other collection of predefined categories.Examples of categories include topical categories such as “sportinggoods,” “electronics,” and “tickets,” and affiliate site “type”categories such as “comparison shopping” and “online coupons.” Asdiscussed below, the affiliates' category selections may be taken intoconsideration in evaluating the performance of, and generatingrecommendations for, specific affiliates and affiliate sites 40. Forexample, the performance of an affiliate web site 40 in the“electronics” category may be compared primarily or exclusively to theperformance of other affiliate sites 40 in the same category.

Although input from the affiliates is helpful to categorizing theaffiliate sites 40, the affiliate sites may additionally oralternatively be categorized or grouped using other methods. Forinstance, a program that crawls and analyzes the content of web sitesmay be used to assign a category identifier to each affiliate oraffiliate site 40 based on located keywords and keyword phrases. Inaddition, a program module may be provided that categorizes theaffiliate sites 40 based on the types of items purchased by the usersthey have referred, and/or based on the types of items to which theyprovide affiliate links 38.

Information about each affiliate, including information collected duringregistration, is stored in an affiliates database 44. This informationmay, for example, include the name, address, affiliate ID, web site URL,affiliate web site category (and/or other site attributes), and bankaccount information of each affiliate. For a relatively large onlinesales site 30, many thousands or hundreds of thousands of differentregistered affiliates may be active at a given time.

Three affiliate web sites 40 are depicted in FIG. 1 for purposes ofillustration. In this example, each affiliate web site 40 includesmultiple item-specific affiliate links 38 to the electronic catalog 34of the online sales site 30. Each such item-specific link 38 istypically displayed on the affiliate site 40 in conjunction with adescription (not shown) of the corresponding catalog item, and typicallypoints to a corresponding item detail within the catalog 34. Eachaffiliate can freely select items from the catalog 34 for which toprovide affiliate links 38 on its respective site 40, and can changethese selections over time. Typically, the set of items selected by agiven affiliate will be very small compared to the number of items inthe catalog 34.

The affiliates may also be permitted to provide affiliate links 38 thatare not specific to, and are not provided in conjunction with adescription of, any particular catalog item. For instance, an affiliateweb site 40 may include an affiliate link 38 that points to the homepage of the online sales web site (as illustrated in FIG. 1), or whichpoints to a page associated with a particular item category or store.Some affiliate sites 40 may only have non-item-specific affiliate links,and no item-specific affiliate links.

Each affiliate link 38, whether specific to a particular catalog item ornot, is tagged with the affiliate ID of the corresponding affiliate.Typically, the affiliate ID is included in a predefined tag locationwithin the target URL of the link 38. The presence of the affiliate IDenables the servers of the commerce system 32 to identify and track thesource of each referral of a user from an affiliate site 40 to theonline sales site 30. This enables the operator of the online sales site30 to pay the referral fees, such as commissions on resulting sales, tothe affiliates that refer users. As discussed below, the tasks oftracking the referrals and paying the referral fees may alternatively beperformed by a separate computer system operated by an affiliate programservice provider.

A user may, but need not, initiate browsing of the electronic catalog 34by selecting a link 38 on an affiliate site. If a user selects anaffiliate link 38 and then makes a purchase from the online sales site30, the sale may be credited or attributed to the correspondingaffiliate, in which case the affiliate may be paid a commission that isbased on the sales price. When the purchase of an item is attributed toa particular affiliate, the affiliate is commonly referred to as having“sold” the item, even though the sale is actually transacted by theonline sales site 30.

The rules for attributing sales to affiliates vary from program toprogram. As one example, the affiliate may be given credit for allpurchases made by the referred user within a particular time period(e.g., twenty-four hours) following the user's selection of theaffiliate link 38, excluding any such purchases made after the userre-enters the online sales site 30 from a site of a different affiliate.As another example, the affiliate may only be given credit if the itempurchased is the same as the item selected by the user while browsingthe affiliate's site 40. As will be apparent, the present invention isnot limited to any particular rule or set of rules for attributing salesto affiliates.

In the embodiment shown in FIG. 1, the rules for attributing salestransactions to affiliates are embodied within a transaction processingengine 48 that handles the online checkout process. Each time a userpurchases an item via the online sales site 30, the transactionprocessing engine 48 records the purchase in a transactions database 50.If a given item purchase is to be attributed to a particular affiliate,the item purchase event is tagged in the database 50 with thecorresponding affiliate ID. An executable component (not shown) uses theaffiliate ID tags and associated transaction data recorded in thedatabase 50 to calculate commissions or other referral fees to be paidto specific affiliates.

As shown in FIG. 1, a performance analysis engine 54 analyzes the taggedtransaction data collected in the transactions database 50 to evaluatethe performance of, and to make recommendations for, specific affiliatesand affiliate sites 40. As shown in FIG. 1, the performance analysisengine 54 may take into consideration the categories or attributes ofthe affiliate web sites 40 as part of this analysis. For example, therecommendations generated for a particular affiliate web site 40 fallingwithin a given category may be based solely on transactions attributableto affiliate sites falling within that category. The performanceanalysis engine 54 may additionally or alternatively generaterecommendations that are based on a broader analysis of the transactiondata, such as an analysis that takes into consideration all transactionstagged with affiliate IDs.

In the embodiment depicted in FIG. 1, the performance analysis engine 54includes four components: an association mining engine 58, and databaseor other repository 60 of association rules, a disparity detectionmodule 62, and a report generation module 64. The association miningengine 58 periodically analyzes transaction data collected over a periodof time, such as the last six weeks or months, to generate a set ofassociation rules that identify items that tend to be purchased incombination. Each association rule may be represented by the expressionX1→X2 (P), where X1 and X2 are mutually exclusive sets of items in thecatalog 34, and P represents the conditional probability that apurchaser of X1 will also purchase X2. For instance, the associationrule {item 1, item 5}→item 3 (65%) indicates that sixty five percent ofthe users who purchase items 1 and 5 also purchase item 3. Typically, X1will have a size of one, two or three, and X2 will have a size of one.An example sequence of steps that may be performed by the associationmining engine 58 to generate the association rules is shown in FIG. 2,which is discussed separately below.

In one embodiment, the association mining engine 58 generates a separateset of association rules for each affiliate web site category—or atleast those categories that include a sufficient number of active sites40 to generate meaningful rules. The association mining engine 58 mayadditionally or alternatively generate association rules that arespecific to a particular affiliate attribute or set of attributes. Forinstance, a set of association rules may be generated for all affiliateswhose transaction volume exceeds a particular threshold, or for allaffiliates whose web sites 40 contain a particular keyword orcombination of keywords. The association mining engine 58 mayadditionally or alternatively generate associate rules that are notspecific to any particular subset of affiliates.

Although association rules are used in the illustrated embodiment, theitem associations detected by the association mining engine 58 may berepresented in a format that does not include association rules. Forexample, the association mining engine 58 may record the detectedassociations in a format in which the conditional probability values areomitted. Thus, the association rules referred to throughout thisdescription may be replaced with other forms of item association data.

In the illustrated embodiment, the association rules generated by theassociation mining engine 58 are stored in a database 60 in associationwith the affiliate site categories or groupings to which theycorrespond. This database 60 is updated over time (e.g., once per month)to reflect new transactions recorded in the transactions database 50.

The disparity detection module 62 uses the association rules stored inthe database 60 to separately analyze the performance of eachaffiliate/affiliate site 40, and to identify significant performancedisparities to call to the attention of the affiliate. As part of thisprocess, the applicable association rules may be used to detectsignificant disparities between expected and actual sales quantities ofparticular items.

For instance, suppose the association rule {item A, item D}→{item F}(50%) has been generated for a particular category of affiliate sites.Suppose further that a particular affiliate in this category has, duringthe past month, sold items A and D to two hundred different users, buthas only sold item F to three users. In this example, a significantdisparity exists between the affiliate's expected sales quantity of0.5×200=100 units of item F, and the actual quantity of three. Thisdisparity may be attributable to the absence, or the poor placement, ofitem F on the affiliate's web site 40. Thus, if item F is recommended tothis affiliate, or if the affiliate is notified of this disparity, theaffiliate can appropriately modify its web site 40 to significantlyimprove its performance in connection with item F.

An example sequence of steps that may be performed by the disparitydetection module 62 is depicted in FIG. 3, and is discussed separatelybelow.

As further depicted in FIG. 1, significant disparities detected by thedisparity detection module 62 are incorporated into affiliate-specificperformance reports generated by a report generation module 64. Theperformance reports may include recommendations of specific items thatcan be added to the affiliate site 40 to improve performance. Oneexample of such a report is shown in FIG. 4, which is discussed below.The reports may be conveyed to the corresponding affiliates as emailmessages, as personalized web pages, by facsimile, and/or using anyother communications method.

In one embodiment, the performance analysis engine 54 is accessible viaan affiliate extranet that can be securely accessed by affiliates tointeractively generate performance reports. The extranet may providefunctionality for an affiliate to interactively select an affiliate sitecategory, and to then view a performance report showing how its site 40is performing within that category. The extranet may also providefunctionality for the affiliate to interactively specifying a date rangeover which to analyze the performance of the affiliate site 40.

II. Generation of Association Rules (FIG. 2)

FIG. 2 illustrates one example of process (sequence of steps) that maybe executed by the association mining engine 58 to generate sets ofassociation rules for each predefined affiliate site category. Thisprocess may be repeated periodically, such as once per month, togenerate a new set of rules for each category.

In step 72 of FIG. 2, all transactions that occurred in the last L daysand are tagged with an affiliate ID are retrieved from the transactionsdatabase 50. The variable L represents a look-back horizon, in days, foranalyzing the transaction data. In step 74, the first category ofaffiliate web sites is selected. In step 76, all transactionsattributable to affiliate sites 40 falling outside the currentlyselected category are filtered out for purposes of the current iterationof the process. The category of a given affiliate site 40 may, forexample, be determined based on any one or more of the following: (a)category information supplied by the affiliate during or followingregistration; (b) the item categories of items purchased by usersreferred by the affiliate site; (c) the item categories associated withthe catalog pages to which users are referred by the affiliate site; (d)the results of a content-based analysis performed by a program thatcrawls and analyzes the content of affiliate sites; (e) categoryassignments made by a human operator who reviews the affiliate sites;(f) any other type of site information that is accessible to theassociation mining engine 58.

In step 78, an association-rule mining algorithm is applied to theremaining transaction data (i.e., to the transactions not filtered outin the current iteration) to generate an initial set of associationrules for the currently-selected category. Any of a variety of wellknown association-rule mining algorithms may be used for this purpose.One algorithm that is well suited for large domains of items is theCHARM algorithm, which is described in M. J. Zaki and C. Hsiao, “CHARM:An efficient algorithm for closed itemset mining,” 2nd SIAM Int'l Conf.on Data Mining, April, 2002, the disclosure of which is herebyincorporated by reference. Other algorithms that may be used includeEclat and Apriori. In step 80, the association rules having conditionalprobability or “confidence” values falling below a particular threshold,such as 0.3 (or 30%), are discarded, and the remaining association rulesare stored in the database 60 in association with the currently selectedcategory.

The association rules generated in step 78 typically include rules thatmap two-item sets to single-item sets. The association rule {item 2,item 4}→item 1 (65%) is one example of a rule that represents such a2-to-1 mapping. Rules that represent 1-to-1 mappings, 3-to-1 mappings,1-to-2 mappings, and 2-to-2 mappings may also be generated and used.

As depicted by blocks 82 and 84 of FIG. 2, steps 76-80 are repeated foreach category of affiliate web site 40 until association rules have beengenerated for all categories. In addition or as an alternative togenerating category-specific association rules, a set of generalassociation rules may be generated based on all of the transaction dataretrieved in step 72. One benefit to generating association rules thatare specific to a particular category or type of affiliate site 40, asopposed to generating a general set of association rules, is that thetotal processing burden is significantly reduced. This is because theset of items purchased by all users referred by a particular category ortype of affiliate tends to be much smaller than the set of itemspurchased by all users referred by affiliate sites.

One attribute of the process shown in FIG. 2 is that the associationrules generated for a particular site category are based solely onpurchases attributable to affiliate sites in that category. As a result,the rules strongly reflect the preferences of users who browse thatparticular category of affiliate site. One variation to the processshown in FIG. 2 is to also take into consideration other purchases madeby these same users. Specifically, the association rules for category Dmay be based on the purchases made by all users referred by category Daffiliate sites 40, including but not limited to purchases that areactually attributed to an affiliate site in category D. One possiblebenefit of this variation is that a greater quantity of transaction datamay be used to generate the association rules; one potentialconsideration is that the rules may fail to reflect users' preferencesto select certain types of items from affiliate sites, and to selectother types of items directly from the online sales site 30.

As mentioned above, although association rules are generated and used inthe illustrated embodiment, other forms of association data mayalternatively be used.

III. Application of Association Rules to Detect Disparities (FIG. 3)

FIG. 3 illustrates a process that may be executed by the disparitydetection module 62 to evaluate the performance of a particularaffiliate site 40. This process may be executed periodically (e.g., onceper month) for each active affiliate site, and/or may be invoked when anaffiliate logs in and initiates generation of a performance report.

In step 92, the transactions database 50 is accessed to retrievetransaction data describing all item purchases made within the last Mdays (e.g., 30 days) that are attributed to this affiliate. In somecases, a given affiliate may operate multiple web sites web sites usingthe same affiliate ID. In these cases, the transaction data retrieved instep 92, and analyzed in the subsequent steps, will reflect the userreferrals from all such web sites 40. Stated differently, the affiliatesite 40 whose performance is analyzed in this situation may be thoughtof as including all web pages that include affiliate links 38 taggedwith the same affiliate ID, even though some of these pages may behosted by servers in different Internet domains than others.

In step 94, the association rules that are applicable to this affiliateare retrieved from the association rules database 60. If this affiliatesite 40 is assigned to a particular category of affiliate sites, theassociation rules retrieved in this step 94 may be the current set ofassociation rules for this category. Otherwise, a generic set ofassociation rules that are applicable to all affiliate sites may beretrieved and used. One benefit to using category-specific associationrules is that the rules tend to be much more applicable to, andreflective of the user preferences of, the affiliate web sites 50 towhich they are applied.

In step 96, the retrieved association rules are applied to thetransaction data retrieved in step 92 to generate expected quantityvalues for specific items. In step 98, these expected quantity valuesare compared to the corresponding actual quantity values to determinewhether any significant disparities exist. A disparity may be treated assignificant if, for example, an item's expected quantity exceeds itsactual quantity by a certain number (e.g., fifty). The ratio between theexpected and actual values may also be taken into consideration indetermining whether the disparity is significant.

The following example illustrates how significant disparities may bedetected in steps 96 and 98. For purposes of this example, assume thatthe following two association rules are applicable to the currentaffiliate:

Rule 1: {item B, item D}→item A (95%)

Rule 2: {item A, item E}→item F (50%)

In addition, assume the following sales activity has been attributed tothis affiliate during the relevant time period:

800 sales of {item B, item D}

30 sales of {item A, item E}

30 sales of item A

10 sales of item F

Applying rule 1 to this activity data produces an expected quantity of0.95×800=760 units of item A. Because this expected quantity greatlyexceeds the actual quantity of sixty, the disparity between the two maybe treated as significant, and may be used as a basis for recommendingthat item A be listed, or displayed more prominently, by this affiliate.It should be noted in this example that although the affiliate has soldsixty units of item A, item A may not actually be listed on theaffiliate's web site 40; this is because a sale of item A may beattributed to the affiliate even though the purchaser entered the onlinesales site 30 by selecting a different item, or by selecting anon-item-specific link, on the affiliate's site. As mentioned above,some affiliate programs may not attribute this type of sale to theaffiliate.

Continuing this example, application of rule 2 to the affiliate'sactivity data produces an expected quantity of 0.50×30=15 units of itemF. Because this value does not significantly exceed the actual quantityof ten, the affiliate's sales of item F may be treated as generallyconsistent with expectations. Hence, rule 2 would not be used as a basisfor recommending item F to this affiliate.

As will be recognized, a variety of other performance metrics may beused to evaluate the performance of each affiliate site 40. For example,an affiliate's performance with a respect to a given catalog item mayadditionally or alternatively be assessed based on (a) the amount ofrevenue generated by this affiliate in comparison to other affiliates inthe same category, and/or (b) the number of users referred to the item'sdetail page by this affiliate in comparison to other affiliates in thesame category. Regardless of the particular performance metric used, theaffiliate's actual performance with respect to a particular item may becompared against the expected performance, where the expectedperformance takes into consideration both (a) this affiliate'sperformance with respect to other items, and (b) the performance ofother affiliates, such as those falling in the same category.

As depicted in block 100, any significant disparities that are detectedare reported to the report generation module 64 (FIG. 1), and areincorporated into a performance report that is transmitted to theaffiliate. For each recommended item, the report generation module may64 generate, and incorporate into the report, a sequence of HTML orother coding for adding an affiliate link that points to the item in theelectronic catalog 34 (see example report in FIG. 4). This link coding,which includes the affiliate's unique ID, may be copied by the affiliateinto an associated HTML document to add the affiliate link to theaffiliate site 40.

IV. Example Performance Report (FIG. 4)

FIG. 4 illustrates one example of a report format that may be used toprovide recommendations to the affiliates. In this example, the reportrecommends three specific products to the affiliate, and contains HTMLcoding for augmenting its affiliate site by adding links to theseproducts. The report also includes a general explanation of why thesethree products are being recommended. A more detailed explanation thatspecifies, for example, the affiliate's actual and expected salesquantities of these products may alternatively be used. Further, ratherthan providing explicit recommendations, the performance report coulddisplay the raw data, and let the affiliate decide whether thedisparities between expected and actual sales quantities aresufficiently significant to justify changes to its web site 40.

As mentioned above, personalized reports of the type shown in FIG. 4 maybe generated automatically (e.g., once per month) and transmitted to theaffiliates by email or other communications method. In addition, anextranet or other interactive system may be provided through whichaffiliates can interactively request and view specific reports.

V. Hardware and Software Components

The components of the performance analysis engine 54 shown in FIG. 1,including the association mining engine 58, the disparity detectionmodule 62, and the report generation module 64, may be implementedwithin program modules executed by one or more general purposecomputers. The functions performed by these components 58, 62, 64 may beintegrated into a common computer program, or may be implemented inseparate computer programs (which may or may not run on the samecomputer). The databases 44, 50, 60 shown in FIG. 1 may be any type ofelectronic repository used to store data, and may be combined with eachother and/or with other databases.

The online sales web site 30 may be implemented with server softwareexecuted by one or more general purpose computers, which may or may notbe the same computers used to implement the performance analysis engine54. The web pages of the site 30 may be encoded using HTML (HypertextMarkup Language) and/or another markup language. Each affiliate web site40 may be implemented by any type of computer system capable of servingweb pages in response to HTTP requests. A given affiliate site mayconsist of a single web page (e.g., a personal page of an individualuser) or a collection of web pages.

Users may browse the online sales site 30 and the affiliate sites 40using any type of computing device that provides web browsingcapabilities (personal computers, PDAs, cell phones, etc.). Typically,each such computing device runs browser software capable of generatingHTTP requests.

The computers used to implement the commerce system 32 may beinterconnected by one or more computer networks, and may, but need not,reside locally to each other. The computers that host the affiliatesites 40 typically reside remotely from those of the commerce system 32.

VI. Content-Based Recommendations

In the embodiments described above, the performance of a given affiliateor affiliate site 40 is evaluated based on the transaction dataassociated with that affiliate or affiliate site. Another approach thatmay additionally or alternatively be used is to programmatically crawland analyze the content of the affiliate site 40 to identify the itemsthat are currently listed. Once this set of items is known,recommendations may be provided of other items that are frequentlypurchased by those who purchase one or more of the listed items. Itemassociation data (e.g., a set of association rules) that is specific tothe corresponding affiliate category may be used for this purpose.

As part of the programmatic analysis of site content, the degrees towhich specific items are separated from each other on the affiliate site40 may be detected and analyzed. For example, if a strong associationexists between items A and B, but the two items are separated by morethan a threshold “distance” on the affiliate site 40 (e.g., by more thanone click, or by more than a threshold number of intervening items on apage on which both items are listed), the performance report may suggestlisting these items in closer proximity.

VII. Externally Managed Affiliate Programs

Many existing affiliate programs are partially or wholly managed by athird party affiliate program service provider, such as CommissionJunction or ValueClick. With these “externally managed” programs, eachaffiliate link 38 to the online sales site 30 may be an indirect linkthat actually points to a separate computer system or site operated bythe service provider. When a user selects such a link 38, the serviceprovider site may record the referral event and transparently redirectthe user's browser to the online sales web site 30. The service providersite may also use the recorded referral information, in combination withtransaction data reported back by the online sales site 30, to calculatereferral fees to be paid to the various affiliates.

With this type of implementation, the performance analysis engine 54(FIG. 1) may be implemented as part of the service provider site, ratherthan as part of the commerce system 30. In this type of embodiment, asingle performance analysis engine 54 may provide performance reports tothe affiliates of many different affiliate programs of many differentonline sales sites 30. The system implementation may otherwise begenerally the same as shown in FIG. 1 and described above.

VIII. Web Service Implementations

The performance analysis engine 54 may also be implemented as a webservice that can be called over the Internet by other computer systems.With this approach, a commerce system 32 that implements its ownaffiliates program can periodically pass sets of transaction data to theweb service, which may use this data to generate and return performancereports of the type described above. The performance reports may then bedelivered to the corresponding affiliates. Affiliate program serviceproviders may also use the web service to obtain performance reports tosend to affiliates.

IX. Recommendations for Other Types of Online Sellers

The system and methods described above can also be used to evaluate theperformance of, and provide recommendations to, online sellers thatactually sell items. The online sellers may, for example, includemerchants that have their own web sites or storefronts within an onlinemall, and/or may include small business entities that sell items in anonline marketplace or on an online auction site.

As in the affiliate-based embodiments described above, the onlinesellers may initially be grouped into categories. This may beaccomplished using one or more of the following: (a) seller-suppliedcategory data, (b) transaction data associated with each online seller,(c) catalog or listing content provided by each online seller. Once theonline sellers have been assigned to specific seller categories, thegeneral process shown in FIG. 2 may be used to generate a separate setof item association rules for each such category. For example, togenerate a set of association rules for the category “sellers of cookingsupplies,” the aggregated sales data of all online sellers within thiscategory may be analyzed using the CHARM algorithm or anotherassociation-rule mining algorithm.

Finally, the process shown in FIG. 3 may be used to evaluate theperformance of, and generate seller-specific performance reports for,particular online sellers. For instance, to evaluate the performance ofa particular online seller in the cooking supplies category, theassociation rules for this category may be applied to the actual salesdata of this particular seller to identify disparities between actualand expected performance. The results of this analysis, which mayinclude recommendations of additional items to sell, may then betransmitted to the online seller.

X. Conclusion

The systems and methods described herein may also be used to make othertypes of recommendations to affiliates. For instance, the performanceanalysis engine 58 may additionally or alternatively recommendcategories or types of items to list on the affiliate site 40. Inaddition, for affiliate programs in which affiliates can selectadvertisements, such as banner ads, to host on their respective sites40, the performance analysis engine 58 may identify and recommendspecific ads to host on a particular affiliate site to improve itsperformance.

Although this invention has been described in terms of certainembodiments and applications, other embodiments and applications thatare apparent to those of ordinary skill in the art, includingembodiments that do not provide all of the features and advantages setforth herein, are also within the scope of this invention. Accordingly,the scope of the present invention is defined only by the appendedclaims.

1. A method of analyzing the performance of sellers, the method comprising: collecting transaction data of a plurality of sellers that sell items online in a seller category, said transaction data identifying specific items sold by the sellers to buyers; generating, based on the transaction data of the plurality of sellers, item association data that identifies items that are purchased in combination relatively frequently by the buyers, said item association data not being specific to any particular seller; analyzing performance of a particular seller by comparing the item association data to transaction data associated with the particular seller; and generating for the particular seller, based on results of the analysis of performance, an electronic report that includes a recommendation of at least one item for the particular seller to sell to improve performance; said method performed in its entirety by a computer system that comprises one or more computers.
 2. The method of claim 1, wherein analyzing the performance of the particular seller comprises detecting disparities between expected and actual sales quantities of specific items.
 3. The method of claim 2, wherein generating the electronic report comprises using said disparities to generate recommendations of items for the particular seller to sell.
 4. The method of claim 1, wherein said item is an item that is not currently sold by the seller.
 5. The method of claim 1, wherein the item association data comprises a plurality of association rules.
 6. The method of claim 5, wherein at least one of the association rules identifies a first set of items, X1, and a second set of items, X2, and specifies a conditional probability that a purchaser of X1 will also purchase X2.
 7. The method of claim 5, wherein at least one of the association rules maps a two-item set to a single item.
 8. The method of claim 1, wherein the item association data comprises data values reflective of how frequently specific items are purchased in combination.
 9. The method of claim 1, further comprising automatically assigning the seller to said seller category based on electronic content provided by the seller.
 10. A computer system configured to analyze the performance of sellers that sell items online, said computer system comprising one or more computers, said computer system programmed to implement: an association mining engine that generates item association data based on transaction data of a plurality of sellers, said transaction data reflective of item sales transactions of the sellers, said item association data identifying items that are purchased in combination relatively frequently by buyers; and an analysis system that uses the item association data, in combination with transaction data of a seller, to perform an analysis of performance of the seller, and which generates, based on results of the analysis, recommendations of items for the seller to sell to improve performance.
 11. The computer system of claim 10, wherein the analysis system uses the item association data in combination with the transaction data of the seller to detect disparities between expected and actual quantities of specific items sold by the seller.
 12. The computer system of claim 11, wherein the analysis system uses the detected disparities to generate said recommendations.
 13. The computer system of claim 10, wherein the association mining engine generates separate sets of item association data for each of a plurality of seller categories, and the analysis system uses each set of item association data to assess the performance of sellers falling in the respective seller category.
 14. The computer system of claim 10, wherein the item association data comprises a plurality of association rules.
 15. The computer system of claim 14, wherein at least one of the association rules identifies a first set of items, X1, and a second set of items, X2, and specifies a conditional probability that a purchaser of X1 will also purchase X2.
 16. Non-transitory computer storage having stored thereon executable code that directs a computer system to perform a process that comprises: storing transaction data of a plurality of sellers that sell items online in a seller category, said transaction data identifying specific items sold by the sellers to buyers; generating, based on the transaction data of the plurality of sellers, item association data that identifies items that are purchased in combination relatively frequently by the buyers, said item association data not being specific to a particular seller; analyzing performance of a particular seller by comparing the item association data to transaction data of the particular seller; and generating for the particular seller, based on results of the analysis of performance, a recommendation of at least one item for the seller to sell to improve performance.
 17. The non-transitory computer storage of claim 16, wherein analyzing the performance of the particular seller comprises detecting disparities between expected and actual sales quantities of specific items.
 18. The non-transitory computer storage of claim 16, wherein the process further comprises generating an electronic report that includes the recommendation, and transmitting the electronic report to the particular seller. 